Deep neural network model of haptic saliency

Haptic exploration usually involves stereotypical systematic movements that are adapted to the task. Here we tested whether exploration movements are also driven by physical stimulus features. We designed haptic stimuli, whose surface relief varied locally in spatial frequency, height, orientation, and anisotropy. In Experiment 1, participants subsequently explored two stimuli in order to decide whether they were same or different. We trained a variational autoencoder to predict the spatial distribution of touch duration from the surface relief of the haptic stimuli. The model successfully predicted where participants touched the stimuli. It could also predict participants' touch distribution from the stimulus' surface relief when tested with two new groups of participants, who performed a different task (Exp. 2) or explored different stimuli (Exp. 3). We further generated a large number of virtual surface reliefs (uniformly expressing a certain combination of features) and correlated the model's responses with stimulus properties to understand the model's preferences in order to infer which stimulus features were preferentially touched by participants. Our results indicate that haptic exploratory behavior is to some extent driven by the physical features of the stimuli, with e.g. edge-like structures, vertical and horizontal patterns, and rough regions being explored in more detail

The many colours of ‘the dress’

YesThere has been an intense discussion among the public about the colour of a dress, shown in a picture posted originally on Tumblr (http://swiked. tumblr.com/post/112073818575/ guys-please-help-me-is-this-dress white-and; accessed on 10:56 am GMT on Tue 24 Mar 2015). Some people argue that they see a white dress with golden lace, while others describe the dress as blue with black lace. Here we show that the question “what colour is the dress?” has more than two answers.The full text was made available at the end of the publisher's embargo, 14th May 201

Unsupervised learning of haptic material properties.

When touching the surface of an object, its spatial structure translates into a vibration on the skin. The perceptual system evolved to translate this pattern into a representation that allows to distinguish between different materials. Here, we show that perceptual haptic representation of materials emerges from efficient encoding of vibratory patterns elicited by the interaction with materials. We trained a deep neural network with unsupervised learning (Autoencoder) to reconstruct vibratory patterns elicited by human haptic exploration of different materials. The learned compressed representation (i.e., latent space) allows for classification of material categories (i.e., plastic, stone, wood, fabric, leather/wool, paper, and metal). More importantly, classification performance is higher with perceptual category labels as compared to ground truth ones, and distances between categories in the latent space resemble perceptual distances, suggesting a similar coding. Crucially, the classification performance and the similarity between the perceptual and the latent space decrease with decreasing compression level. We could further show that the temporal tuning of the emergent latent dimensions is similar to properties of human tactile receptors

Foveal to peripheral extrapolation of facial emotion.

Peripheral vision is characterized by poor resolution. Recent evidence from brightness perception suggests that missing information is filled out with information at fixation. Here we show a novel filling-out mechanism: when participants are presented with a crowd of faces, the perceived emotion of faces in peripheral vision is biased towards the emotion of the face at fixation. This mechanism is particularly important in social situations where people often need to perceive the overall mood of a crowd. Some faces in the crowd are more likely to catch people's attention and be looked at directly, while others are only seen peripherally. Our findings suggest that the perceived emotion of these peripheral faces, and the overall perceived mood of the crowd, is biased by the emotions of the faces that people look at directly

Three perceptual dimensions for specular and diffuse reflection

Previous research investigated the perceptual dimensionality of achromatic reflection of opaque surfaces, by using either simple analytic models of reflection, or measured reflection properties of a limited sample of materials. Here we aim to extend this work to a broader range of simulated materials. In a first experiment, we used sparse multidimensional scaling techniques to represent a set of rendered stimuli in a perceptual space that is consistent with participants’ similarity judgments.Participants were presented with one reference object and four comparisons, rendered with different material properties.They were asked to rank the comparisons according to their similarity to the reference, resulting in an efficient collection of a large number of similarity judgments. In order to interpret the space individuated by multidimensional scaling, we ran a second experiment in which observers were asked to rate our experimental stimuli according to a list of 30 adjectives referring to their surface reflectance properties. Our results suggest that perception of achromatic reflection is based on at least three dimensions, which we labelled “Lightness”, “Gloss” and “Metallicity”, in accordance with the rating results. These dimensions are characterized by a relatively simple relationship with the parameters of the physically based rendering model used to generate our stimuli, indicating that they correspond to different physical properties of the rendered materials. Specifically,“Lightness” relates to diffuse reflections, “Gloss” to the presence of high contrast sharp specular highlights and “Metallicity” to spread out specular reflections

Perception of saturation in natural objects

The distribution of colors across a surface depends on the interaction between its surface properties, its shape, and the lighting environment. Shading, chroma, and lightness are positively correlated: points on the object that have high luminance also have high chroma. Saturation, typically defined as the ratio of chroma to lightness, is therefore relatively constant across an object. Here we explored to what extent this relationship affects perceived saturation of an object. Using images of hyperspectral fruit and rendered matte objects, we manipulated the lightness–chroma correlation (positive or negative) and asked observers which of two objects appeared more saturated. Despite the negative-correlation stimulus having greater mean and maximum chroma, lightness, and saturation than the positive, observers overwhelmingly chose the positive as more saturated. This suggests that simple colorimetric statistics do not accurately represent perceived saturation of objects—observers likely base their judgments on interpretations about the cause of the color distribution

Target Search and Inspection Strategies in Haptic Search

Haptic search is a common everyday task, usually consisting of two processes: target search and target analysis. During target search we need to know where our fingers are in space, remember the already completed path and the outline of the remaining space. During target analysis we need to understand whether the detected potential target is the desired one. Here we characterized dynamics of exploratory movements in these two processes. In our experiments participants searched for a particular configuration of symbols on a rectangular tactile display. We observed that participants preferentially moved the hand parallel to the edges of the tactile display during target search, which possibly eased orientation within the search space. After a potential target was detected by any of the fingers, there was higher probability that subsequent exploration was performed by the index or the middle finger. At the same time, these fingers dramatically slowed down. Being in contact with the potential target, the index and the middle finger moved within a smaller area than the other fingers, which rather seemed to move away to leave them space. These results suggest that the middle and the index finger are specialized for fine analysis in haptic search

Deep neural network model of haptic saliency

Haptic exploration usually involves stereotypical systematic movements that are adapted to the task. Here we tested whether exploration movements are also driven by physical stimulus features. We designed haptic stimuli, whose surface relief varied locally in spatial frequency, height, orientation, and anisotropy. In Experiment 1, participants subsequently explored two stimuli in order to decide whether they were same or different. We trained a variational autoencoder to predict the spatial distribution of touch duration from the surface relief of the haptic stimuli. The model successfully predicted where participants touched the stimuli. It could also predict participants’ touch distribution from the stimulus’ surface relief when tested with two new groups of participants, who performed a different task (Exp. 2) or explored different stimuli (Exp. 3). We further generated a large number of virtual surface reliefs (uniformly expressing a certain combination of features) and correlated the model’s responses with stimulus properties to understand the model’s preferences in order to infer which stimulus features were preferentially touched by participants. Our results indicate that haptic exploratory behavior is to some extent driven by the physical features of the stimuli, with e.g. edge-like structures, vertical and horizontal patterns, and rough regions being explored in more detail

Context effects on the perception of saturation of fruit colors in still-life paintings.

Still-life painters, especially of the so-called Golden Age (17th century) in the Netherlands, are famous for their masterful techniques of rendering reality. Their amazing abilities to depict different material properties of fruits and flowers are stunning. But how important are these careful arrangements of different objects for the perception of an individual item? Is the perceived color saturation of a single fruit influenced by its surrounding context? We selected fruits in still-life paintings as stimuli to investigate whether and how perceived saturations of fruits were affected by their original contexts. In our study, we focused especially on effects of five context properties: complementary colors, chromatic and luminance contrast, object overlap, and surround variance. Six fruit varieties depicted in high-quality digital reproductions of 48 classic and eight varieties in 64 more recent, modern still-life paintings were selected. In a single trial, eight images of fruits of the same variety appeared on a neutral gray background; half were single fruit cutouts, and the other half were the same fruits embedded in their circular contexts. Fifteen participants ranked all eight images according to perceived color saturations of the fruits. Saturation ratings showed a high agreement of 77%. Surrounding contexts led to an increase in perceived saturation of central fruits. This effect was mainly driven by object overlap, the presence of the central fruit type also in the context, and surround variance. Chroma contrast between fruits and contexts decreased saturation significantly. No significant context effects were found for complementary colors or luminance contrast. Our results show that in paintings, many of the cues that are usually experimentally isolated occur in interesting combinations and lead to an increase in perceived saturation that makes fruit objects more appealing and convincing